Display devices which are intended to provide an immersive experience normally allow a user to turn his head and experience a corresponding change in the scene which is displayed. Head mounted displays sometimes support 360 degree viewing in that a user can turn around while wearing a head mounted display with the scene being displayed changing as the user's head position is changes.
In order to support 360 degrees of view, a 360 degree scene may be captured using multiple cameras, e.g., with multiple stereoscopic camera pairs or individual mono cameras, with the images being combined to generate the 360 degree scene which is to be made available for viewing.
Given transmission the constraints, e.g., network data constraints, associated with content being streamed, it may not be possible to stream the full 360 degree view in full high definition video to all customers seeking to receive and interact with the content. This is particularly the case where the content is stereoscopic content including image content intended to correspond to left and right eye views to allow for a 3D viewing effect.
In the case of stereoscopic camera rigs, wide angle lenses, e.g., fisheye camera lenses, may be used to capture a wide viewing area.
While fisheye lenses may capture a wide viewing area, the captured image may only occupy a portion of a frame due to the way a fisheye lens directs light onto a sensor. This can result in portions of a frame being encoded with no useful image content when the captured images are encoded as they are and sent to a playback device.
Efficient use of available bandwidth is a technical problem that remains to be addressed. This is particularly the case where fish eye lenses are used to capture images to be communicated. Given the limited bandwidth available for communicating content to a playback device it would be desirable if methods and/or apparatus could be developed which could efficiently communicate images and/or other content to a playback device without wasting bandwidth that might normally be used to communicate blank portions of a frame that is used to communicate an image captured using a fisheye lens.
As part of making efficient use of available bandwidth used to communicate a frame, it would be desirable if in at least some embodiments portions of a frame could be used to communicate additional image content, e.g., content captured by a different camera than the camera used to capture an image communicated in a large portion of the frame and content that is provided by another source of image content. In addition, while not necessary for all embodiments it would be desirable if at least some embodiments could be implemented using standard encoders and/or if at least some embodiments allowed for information about additional content or its use to be communicated to a playback device.
In addition to efficient use of bandwidth and/or data constraints on encoded image content existing playback systems normally simply decode and display a single image that is received in a frame. How to efficiently and/or effectively communicate image content and/or control use of a playback device which receives image content is a technical problem which needs to be addressed particularly where image content from multiple different sources is communicated in a single frame with different image content to be treated differently by a playback device. There is a need for methods and/or apparatus that allow a playback device to use different portions of a frame differently with potentially some content not being displayed at given times and/or content communicated in a frame being combined or replaced with other content communicated in the same frame during playback. To support such functionality there is a need for methods and/or apparatus which not only allow image content to be communicated by also information on how to use the content and/or other information which is important to rendering images using communicated image content.
In view of the above discussion it should be appreciated that there is a need for methods and apparatus for receiving, transmitting and/or using image content from one or more cameras along with additional content in a frame.